Refactor DSPy-driven optimization flow with coordinator-based orchestration - #48
Open
sandlbn wants to merge 9 commits into
Open
Refactor DSPy-driven optimization flow with coordinator-based orchestration#48sandlbn wants to merge 9 commits into
sandlbn wants to merge 9 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the DSPy-driven kernel optimization flow to be template-driven and adds a new coordinator-based orchestration strategy, while consolidating shared agent utilities and updating pipeline wiring/config to support the new strategy.
Changes:
- Introduces
CoordinatorAgent+ coordinator tools to drive dynamic analyze/apply/benchmark/profile loops viadspy.ReActV2. - Moves signature instruction content into reusable Jinja2 templates and injects them via
append_instructions()across analyzer/planner/optimizer agents. - Updates pipeline strategy wiring, shared utilities, and result metadata (token usage), and bumps the project version/dependency.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/xe_forge/prompts/templates/sycl_optimization_signature.md.j2 | Adds SYCL/CUTLASS optimization instruction template. |
| src/xe_forge/prompts/templates/sycl_algorithmic_signature.md.j2 | Adds SYCL algorithmic optimization instruction template. |
| src/xe_forge/prompts/templates/planning_signature.md.j2 | Adds planning/stage-ordering instruction template. |
| src/xe_forge/prompts/templates/optimization_signature.md.j2 | Adds templated optimizer-stage guidance (device-conditional). |
| src/xe_forge/prompts/templates/optimization_react_signature.md.j2 | Adds templated ReAct optimizer guidance (device-conditional). |
| src/xe_forge/prompts/templates/coordinator_signature.md.j2 | Adds coordinator orchestration instruction template. |
| src/xe_forge/prompts/templates/autotune_signature.md.j2 | Adds autotune instruction template (device-conditional). |
| src/xe_forge/prompts/templates/analysis_signature.md.j2 | Adds analysis instruction template including injected issue-category blocks. |
| src/xe_forge/prompts/templates/algorithmic_signature.md.j2 | Adds algorithmic optimization instruction template. |
| src/xe_forge/prompts/device_prompts.py | Adds template rendering + device addendum helpers for signatures. |
| src/xe_forge/prompts/init.py | Adds Jinja2 environment and render_signature_instructions() export. |
| src/xe_forge/planner.py | Refactors planner to inject template-based planning instructions. |
| src/xe_forge/pipeline.py | Wires coordinator strategy, template-based instructions, token usage tracking, and helper refactors. |
| src/xe_forge/models.py | Adds token_usage field to OptimizationResult. |
| src/xe_forge/config.py | Expands agent strategy comment to include coordinator. |
| src/xe_forge/cli.py | Updates LiteLLM HTTP clients configuration for DSPy setup. |
| src/xe_forge/agents/utils.py | Introduces shared utilities (extract_gemm_dims, verify_sycl, SUCCESS_MESSAGE). |
| src/xe_forge/agents/react_agent.py | Refactors ReAct agent to use shared utilities, template instructions, and ReActV2-style history handling. |
| src/xe_forge/agents/optimizer_agent.py | Refactors optimizer agent to use shared utilities and injected template instructions. |
| src/xe_forge/agents/cover.py | Adjusts CoVeR fallback signature and extraction predictor behavior. |
| src/xe_forge/agents/coordinator.py | Adds CoordinatorAgent implementation using dspy.ReActV2 and coordinator tools. |
| src/xe_forge/agents/coordinator_tools.py | Adds CoordinatorState and tool closures for analyze/retrieve/apply/benchmark/profile/status. |
| src/xe_forge/agents/analyzer_agent.py | Injects template-based analysis guidance and device/DSL naming helpers. |
| src/xe_forge/agents/init.py | Exports CoordinatorAgent from the agents package. |
| src/xe_forge/init.py | Switches to lazy import for XeForgePipeline. |
| pyproject.toml | Bumps version and updates DSPy dependency/source configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR refactors the DSPy integration to make optimization flow more modular, template-driven, and agentic.